Methods and systems for determining system capacity

ABSTRACT

Methods and system are provided for determining a system capacity. The system capacity may be determined by performing one or more capacity tests. The one or more capacity tests may produce capacity test results.

CROSS REFERENCE TO RELATED PATENT APPLICATIONS

This application is a continuation of U.S. application Ser. No.16/268,281, filed on Feb. 5, 2019, which is a continuation of U.S.application Ser. No. 14/795,333, filed on Jul. 9, 2015, and granted asU.S. Pat. No. 10,248,530 on Apr. 2, 2019, each of which are incorporatedby reference in their entireties herein.

BACKGROUND

Current system capacity tests require a large number of iterations fordifferent types of capacity tests. The large number of iterationsincreases testing time. Moreover, results of each type of capacity testcan only provide a one-dimensional perspective on system capacity. Inaddition, traditional capacity tests are often associated withtransaction capacity, rather than user capacity. There is a need formethods and systems to determine system capacity in terms of transactioncapacity and user capacity more efficiently. These and othershortcomings are addressed in the present disclosure.

SUMMARY

It is to be understood that both the following general description andthe following detailed description are exemplary and explanatory onlyand are not restrictive, as claimed. Provided are methods and systemsfor determining a system capacity (e.g., user capacity, transactioncapacity) of a system using any transactional application such as via aserver, a website, banking service, encoding service, decoding service,and/or the like. Systems with transactional applications can have asystem requirement or a system threshold (e.g., service level agreement(SLA)) which can define performance expectations of the system such asminimum speed at which the system will run, reliability of the system,scalability of the system, and the like. The number of transactions onthe system, the rate at which the transactions are occurring, the numberof users, and the like, can affect performance of the system and at somelevel and/or threshold cause the system to be in noncompliance with thesystem requirement.

In an aspect, a system capacity can be determined. The system capacitycan be a threshold of the system at which the performance of the systemstill complies with performance expectations of a system requirement. Asystem capacity can be determined by performing a plurality of capacitytests (e.g., spike tests, transient tests, soak tests, steady statetests). Each capacity test produces capacity test results that can beused to define a region of system capacity. The region of systemcapacity can be an area bound by the plurality of capacity test resultswhen the plurality of capacity test results is graphed. The region ofsystem capacity can comprise system capacity candidates from which thesystem capacity can be selected and still comply with the systemrequirement but allow for maximum performance. In an aspect, the regionof system capacity can be defined by a first set of capacity testresults by performing a first set of capacity tests (e.g., steady statetest) of the system. In an aspect, the first set of capacity tests candetermine a maximum number of transactions on the system (e.g., capacityceiling) with a minimum ramp up rate. The region of system capacity canbe further defined by a second set of capacity test results that can bedetermined by performing a second set of capacity tests (e.g., spiketest) of the system. In an aspect, the second set of capacity tests candetermine the maximum number of transactions the system can perform inthe least amount of time (e.g., capacity floor). The region of systemcapacity can be further defined by a third set of capacity test resultsthat can be determined by performing a third set of capacity tests(e.g., transient test) of the system. In an aspect, the third set ofcapacity tests can determine a maximum ramp up rate that can reach thecapacity ceiling. These capacity tests, when plotted on a graph, candefine the region of system capacity. In an aspect, the system capacitycan be described in terms of a transaction capacity.

In an aspect, one or more of the first set of capacity tests, the secondset of capacity tests, and the third set of capacity tests can becalibrated based on the first set of capacity test results, the secondset of capacity test results, and the third set of capacity testresults. For example, the second set of capacity test results might beinconsistent with the first set of capacity tests results and thereforethe system can be adjusted to get a more accurate first set of capacitytest results and/or second set of capacity test results.

In an aspect, a fourth set of capacity tests can be performed to confirmthe region of system capacity and/or further define the region of systemcapacity by using one or more respective capacity test results of thefirst set of capacity tests, the second set of capacity tests, and thethird set of capacity tests. In an aspect, the fourth set of capacitytests can be performed using one or more of the first set of capacitytest results, the second set of capacity test results, and/or the thirdset of capacity test results. For example, the fourth capacity test canuse the maximum ramp up rate of the third set of capacity tests to reachthe capacity ceiling of the first set of capacity tests and remain atthe capacity ceiling for an extended amount of time to determine whetherthe performance of the system will still be in compliance with thesystem requirement. The region of system capacity determined as a resultof the test results can be used to select a system capacity. The systemcapacity can be selected from the region of system capacity based onperformance parameters defined by the system requirement (e.g. SLA).

In an aspect, the system capacity defined in terms of transactioncapacity can be used to determine a system capacity in terms of a usercapacity. The user capacity can indicate a maximum number of users thatcan be active at any given time on the system and performance of thesystem still complies with the system requirement, or any otherthreshold or network requirement. To determine the user capacity, aplurality of user events can be retrieved from user event logs from aquantity of active users that have been using the system for a giventime period. A user event can be a user command and/or action on thesystem. Each user event can be placed in predefined categories of userevents. User event distribution data can be determined based on thequantity of active users and the plurality of user events associatedwith the quantity of active users. In an aspect, each user event cancorrespond to one or more transactions. For example, a “one timerecording” user event on a DVR system can result in multipletransactions on the system such as “record one shot,” “get recordings,”and “get recordings list.” In an aspect, a user event transactionsequence for each of the quantity of active users can be determined. Theuser event transaction sequence can indicate a sequence of transactionsfor each user event category. The user event transaction sequence canalso indicate a key transaction, which is a transaction that representsthe user event. In an aspect, transaction distribution data can begenerated based on the user event distribution data and the user eventtransaction sequence. The transaction distribution data can be generatedby multiplying the number of user events in the user event distributiondata by each user events' corresponding transactions, including the keytransaction, in the user event transaction sequence. Each transactioncan be sorted in predefined categories. In an aspect, the user capacityof the system (e.g., a maximum number of users associated with thesystem) can be determined based on the transaction distribution data.Each transaction of the transaction distribution data can be divided bythe total number of transactions to determine a percentage weight ofeach transaction. The percentage weight can be multiplied by atransaction capacity determined by capacity tests to determine how manyof the transactions can be performed at a given time. The keytransactions of the transactions can be added together which can be anestimate an of the user capacity of the system.

Additional advantages will be set forth in part in the description whichfollows or may be learned by practice. The advantages will be realizedand attained by means of the elements and combinations particularlypointed out in the appended claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of this specification, illustrate embodiments and together with thedescription, serve to explain the principles of the methods and systems:

FIG. 1 illustrates various aspects of an exemplary system;

FIG. 2 is a block diagram illustrating an example computing device;

FIG. 3 is a flowchart illustrating an example method;

FIG. 4 is a graph illustrating capacity test results and capacitydetermination;

FIG. 5 is a flowchart illustrating an example method; and

FIG. 6 is a flowchart illustrating an example method.

DETAILED DESCRIPTION

Before the present methods and systems are disclosed and described, itis to be understood that the methods and systems are not limited tospecific methods, specific components, or to particular implementations.It is also to be understood that the terminology used herein is for thepurpose of describing particular embodiments only and is not intended tobe limiting.

As used in the specification and the appended claims, the singular forms“a,” “an,” and “the” include plural referents unless the context clearlydictates otherwise. Ranges may be expressed herein as from “about” oneparticular value, and/or to “about” another particular value. When sucha range is expressed, another embodiment includes from the oneparticular value and/or to the other particular value. Similarly, whenvalues are expressed as approximations, by use of the antecedent“about,” it will be understood that the particular value forms anotherembodiment. It will be further understood that the endpoints of each ofthe ranges are significant both in relation to the other endpoint, andindependently of the other endpoint.

“Optional” or “optionally” means that the subsequently described eventor circumstance may or may not occur, and that the description includesinstances where said event or circumstance occurs and instances where itdoes not.

Throughout the description and claims of this specification, the word“comprise” and variations of the word, such as “comprising” and“comprises,” means “including but not limited to,” and is not intendedto exclude, for example, other components, integers or steps.“Exemplary” means “an example of” and is not intended to convey anindication of a preferred or ideal embodiment. “Such as” is not used ina restrictive sense, but for explanatory purposes.

Disclosed are components that can be used to perform the disclosedmethods and systems. These and other components are disclosed herein,and it is understood that when combinations, subsets, interactions,groups, etc. of these components are disclosed that while specificreference of each various individual and collective combinations andpermutation of these may not be explicitly disclosed, each isspecifically contemplated and described herein, for all methods andsystems. This applies to all aspects of this application including, butnot limited to, steps in disclosed methods. Thus, if there are a varietyof additional steps that can be performed it is understood that each ofthese additional steps can be performed with any specific embodiment orcombination of embodiments of the disclosed methods.

The present methods and systems may be understood more readily byreference to the following detailed description of preferred embodimentsand the examples included therein and to the Figures and their previousand following description.

As will be appreciated by one skilled in the art, the methods andsystems may take the form of an entirely hardware embodiment, anentirely software embodiment, or an embodiment combining software andhardware aspects. Furthermore, the methods and systems may take the formof a computer program product on a computer-readable storage mediumhaving computer-readable program instructions (e.g., computer software)embodied in the storage medium. More particularly, the present methodsand systems may take the form of web-implemented computer software. Anysuitable computer-readable storage medium may be utilized including harddisks, CD-ROMs, optical storage devices, or magnetic storage devices.

Embodiments of the methods and systems are described below withreference to block diagrams and flowchart illustrations of methods,systems, apparatuses and computer program products. It will beunderstood that each block of the block diagrams and flowchartillustrations, and combinations of blocks in the block diagrams andflowchart illustrations, respectively, can be implemented by computerprogram instructions. These computer program instructions may be loadedonto a general purpose computer, special purpose computer, or otherprogrammable data processing apparatus to produce a machine, such thatthe instructions which execute on the computer or other programmabledata processing apparatus create a means for implementing the functionsspecified in the flowchart block or blocks.

These computer program instructions may also be stored in acomputer-readable memory that can direct a computer or otherprogrammable data processing apparatus to function in a particularmanner, such that the instructions stored in the computer-readablememory produce an article of manufacture including computer-readableinstructions for implementing the function specified in the flowchartblock or blocks. The computer program instructions may also be loadedonto a computer or other programmable data processing apparatus to causea series of operational steps to be performed on the computer or otherprogrammable apparatus to produce a computer-implemented process suchthat the instructions that execute on the computer or other programmableapparatus provide steps for implementing the functions specified in theflowchart block or blocks.

Accordingly, blocks of the block diagrams and flowchart illustrationssupport combinations of means for performing the specified functions,combinations of steps for performing the specified functions and programinstruction means for performing the specified functions. It will alsobe understood that each block of the block diagrams and flowchartillustrations, and combinations of blocks in the block diagrams andflowchart illustrations, can be implemented by special purposehardware-based computer systems that perform the specified functions orsteps, or combinations of special purpose hardware and computerinstructions.

Provided are methods and systems for determining a system capacity of asystem. The system capacity can comprise one or more of a transactionload capacity, a user capacity, a connection capacity, a transactionrate capacity, and the like. In an aspect, the disclosed methods andsystems can interlock a plurality of capacity test results (e.g., usingcapacity test results to define common regions) from different types ofcapacity tests (e.g., a steady state test, a spike test, a transienttest, and a soak test) to define a region from which the system capacitycan be selected. The disclosed methods and systems can decrease thenumber of iterations and time taken to perform the capacity tests andincrease data accuracy for the capacity test results. Furthermore, oneor more sets of capacity tests can be calibrated based on test resultsof the one or more of the capacity tests. One or more testing parameters(e.g., transaction load, ramp up rate) associated with the calibratedone or more capacity tests can then be used to determine the systemcapacity (e.g., transaction capacity).

In an aspect, the system capacity defined in terms of transactioncapacity can be used to determine the system capacity in terms of a usercapacity. To determine the user capacity, a plurality of user events canbe received for a given time period. A user event can be a user commandand/or action on the system. Each user event can be placed in predefinedcategories of user events. User event distribution data can bedetermined based on the quantity of active users and the plurality ofuser events associated with the quantity of active users. In an aspect,each user event can correspond to one or more transactions. For example,a “one time recording” user event on a digital video recording (DVR)system can result in multiple transactions on the system such as “recordone shot,” “get recordings,” and “get recordings list.” In an aspect, auser event transaction sequence for each of the quantity of active userscan be determined. The user event transaction sequence can indicate asequence of transactions for each user event category. The user eventtransaction sequence can also indicate a key transaction, which is atransaction that represents the user event. In an aspect, transactiondistribution data can be generated based on the user event distributiondata and the user event transaction sequence. The transactiondistribution data can be generated by multiplying the number of userevents in the user event distribution data by each user event'scorresponding transactions, including the key transaction, in the userevent transaction sequence. In an aspect, the user capacity of thesystem (e.g., a maximum number of users associated with the system) canbe determined based on the transaction distribution data. Eachtransaction of the transaction distribution data can be divided by thetotal number of transactions to determine a percentage weight of eachtransaction. The percentage weight can be multiplied by a transactioncapacity determined by capacity tests to determine how many of thetransactions can be performed at a given time. The key transactions ofthe transactions can be added together which can be an estimate an ofthe user capacity of the system.

The disclosed methods and systems can provide a holistic approach tointerpret capacity test results of a system using any transactionalapplications. Examples of transactional applications can include abanking application, a website, and any service such as a DVR service,an encoding service, a decoding service, or other types of service.

FIG. 1 illustrates various aspects of an exemplary system in which thepresent methods and systems can operate. Those skilled in the art willappreciate that present methods may be used in systems that employ bothdigital and analog equipment. One skilled in the art will appreciatethat provided herein is a functional description and that the respectivefunctions can be performed by software, hardware, or a combination ofsoftware and hardware.

The present disclosure is relevant to systems and methods for providingservices to a device, for example, a user device such as a computer,tablet, mobile device, communications terminal, or the like. In anaspect, one or more network devices can be configured to provide variousservices to one or more devices, such as devices located at or near apremises. In another aspect, the network devices can be configured torecognize an authoritative device for the premises and/or a particularservice or services available at the premises. As an example, anauthoritative device can be configured to govern or enable connectivityto a network, such as the Internet or other remote resources, provideaddress and/or configuration services like DHCP, and/or provide namingor service discovery services for a premises, or a combination thereof.Those skilled in the art will appreciate that the present methods may beused in various types of networks and systems that employ both digitaland analog equipment. One skilled in the art will appreciate thatprovided herein is a functional description and that the respectivefunctions can be performed by software, hardware, or a combination ofsoftware and hardware.

The system can comprise a user device 102 in communication with acomputing device 104. By way of example, the user device 102 cancomprise a computer, a PDA, a smartphone, a GPS, a vehicle entertainmentsystem, portable media player, a server, a network device, a set topbox, or other device capable of communicating with the computing device104.

In an aspect, the computing device 104 can be a server for communicatingwith the user device 102. As an example, the computing device 104 cancommunicate with the user device 102 for providing data and/or services.As an example, the computing device 104 can provide one or more servicessuch as a network printing service, a media management service (e.g.,media server), a content service, a DVR service, a streaming service, anencoding service, a decoding service, an encryption service, adecryption service, and the like. In an aspect, the computing device 104can allow the user device 102 to interact with remote resources such asdata, devices, and files. As an example, the computing device 104 can beconfigured as (or disposed at) a central location (e.g., a headend, orprocessing facility), which can receive content (e.g., data, inputprogramming) from multiple sources. The computing device 104 can combinethe content from the multiple sources and can distribute the content touser (e.g., subscriber) locations via a distribution system.

The computing device 104 can be disposed locally or remotely relative tothe user device 102. As an example, the user device 102 and thecomputing device 104 can be in communication via a private and/or publicnetwork 105 such as the Internet or a local area network. Other forms ofcommunications can be used such as wired and wireless telecommunicationchannels, for example.

In an aspect, the user device 102 can comprise a communication element106 for providing an interface to a user to interact with the userdevice 102 and/or the computing device 104. The communication element106 can be any interface for presenting and/or receiving informationto/from the user, such as user feedback. An example interface may becommunication interface such as a web browser (e.g., Internet Explorer,Mozilla Firefox, Google Chrome, Safari, or the like). Other software,hardware, and/or interfaces can be used to provide communication betweenthe user and one or more of the user device 102 and the computing device104. As an example, the communication element 106 can request or queryvarious files from a local source and/or a remote source. As a furtherexample, the communication element 106 can transmit data to a local orremote device such as the computing device 104. For example, the userdevice 102 can transmit a service request (e.g., a DVR search request,one time recording request) to the computing device 104 via thecommunication element 106.

In an aspect, the user device 102 can be associated with a useridentifier or device identifier 108. As an example, the deviceidentifier 108 can be any identifier, token, character, string, or thelike, for differentiating one user or user device (e.g., user device102) from another user or user device. In a further aspect, the deviceidentifier 108 can identify a user or user device as belonging to aparticular class of users or user devices. As a further example, thedevice identifier 108 can comprise information relating to the userdevice such as a manufacturer, a model or type of device, a serviceprovider associated with the user device 102, a state of the user device102, a locator, and/or a label or classifier. Other information can berepresented by the device identifier 108. As another example, the deviceidentifier 108 can be transmitted when a service request (e.g., a DVRservice request, one time recording request) is submitted from the userdevice 102 to the computing device 104.

In an aspect, the device identifier 108 can comprise an address element110 and a service element 112. In an aspect, the address element 110 cancomprise or provide an internet protocol address, a network address, amedia access control (MAC) address, an Internet address, or the like. Asan example, the address element 110 can be relied upon to establish acommunication session between the user device 102 and the computingdevice 104 or other devices and/or networks. As a further example, theaddress element 110 can be used as an identifier or locator of the userdevice 102. In an aspect, the address element 110 can be persistent fora particular network. As an example, the address element 110 can betransmitted when a service request (e.g. one time recording request) issubmitted from the user device 102 to the computing device 104.

In an aspect, the service element 112 can comprise an identification ofa service provider associated with the user device 102 and/or with theclass of user device 102. The class of the user device 102 can berelated to a type of device, a capability of the device, a type ofservice being provided (e.g., an encoding service, a decoding service, atranscoding service, a DVR service), and/or a level of service (e.g.,business class, service tier, service package, and the like). As anexample, the service element 112 can comprise information relating to orprovided by a service provider (e.g., DVR service provider) that isproviding or enabling data flow such as DVR services to the user device102. As a further example, the service element 112 can compriseinformation relating to a preferred service provider for one or moreparticular services relating to the user device 102. In an aspect, theaddress element 110 can be used to identify or retrieve data from theservice element 112, or vice versa. As a further example, one or more ofthe address element 110 and the service element 112 can be storedremotely from the user device 102 and retrieved by one or more devicessuch as the user device 102 and the computing device 104. Otherinformation can be represented by the service element 112.

In an aspect, the computing device 104 can manage the communicationbetween the user device 102 and a database 114 for sending and receivingdata there between. As an example, the database 114 can store aplurality of files (e.g., user event history information), useridentifiers or records, or other information. As a further example, theuser device 102 can request and/or retrieve a file from the database114. In an aspect, the database 114 can store information relating tothe user device 102 such as the address element 110 and/or the serviceelement 112. As an example, the computing device 104 can obtain thedevice identifier 108 from the user device 102 and retrieve informationfrom the database 114 such as the address element 110 and/or the serviceelements 112. As a further example, the computing device 104 can obtainthe address element 110 from the user device 102 and can retrieve theservice element 112 from the database 114, or vice versa. Anyinformation can be stored in and retrieved from the database 114. As anexample, the database 114 can comprise a user event history 116. In anaspect, a user event can be associated with a user operation. The userevent history 116 can comprise user operation history information, suchas time to submit a service request, time to process a service request,and type of the service request associated with a plurality of users(e.g., user device 102). In an aspect, the user event history 116 cancomprise statistical data on user event history information. Forexample, statistical data can comprise total number of active users at agiven time period. As another example, statistical data can compriseuser event distribution at a given time period. A category can be aclass of user events. Using an example of a website, when a userperforms a user event such as selecting a link to navigate to anotherwebpage, a category for the user event can be “link selection.” Eachuser event where a link is selected, regardless of which link isselected, can fall under the category of “link selection.” In an aspect,the user event history 116 can be used to determine a capacity (e.g.transaction capacity, user capacity) of a system (e.g., computing device104). The database 114 can be disposed remotely from the computingdevice 104 and accessed via direct or indirect connection. The database114 can be integrated with the computing device 104 or some other deviceor system.

In an aspect, the computing device 104 can comprise a capacitydetermination element 118. The capacity determination element 118 can beconfigured to determine a system capacity, such as a transactioncapacity of a system, a user capacity of a system, and/or the like. Thetransaction capacity can be a threshold number of transaction beingperformed on the system at which the performance of the system stillcomplies with performance expectations of a system requirement such asdefined by a service level agreement (SLA). The user capacity canindicate a maximum number of users that can be active at any given timeon the system and the performance of the system still complies with thesystem requirement.

In an aspect, the capacity determination element 118 can be configuredto determine a first set of capacity test results, a second set ofcapacity test results, and a third set of capacity test results byperforming a first set of capacity tests (e.g., a steady state test), asecond set of capacity tests (e.g., a spike test), and a third set ofcapacity tests (e.g., a transient test), respectively, on a system. Thefirst set of capacity tests can indicate a maximum number oftransactions (e.g., a capacity ceiling) on the system. The second set ofcapacity tests can indicate a maximum number of transactions on thesystem in the least amount of time (e.g., a capacity floor). The thirdset of capacity tests can indicate a ramp up rate of transactions thatcan achieve the capacity ceiling in the least amount of time. The firstset of capacity tests, the second set of capacity tests, and the thirdset of capacity tests are explained in further detail below.

The capacity determination element 118 can be configured to calibratecapacity tests based on one or more capacity test results. For example,the capacity determination element 118 can be configured to calibrateone or more of the first set of capacity tests, the second set ofcapacity tests, the third set of capacity tests, and/or the like. In anaspect, calibration can be performed based on the first set of capacitytest results, the second set of capacity test results, the third set ofcapacity test results, and/or the like. Calibration can be performed bycomparing the first set of capacity test results, the second set ofcapacity test results, and the third set of capacity test results toeach other to determine whether or not there are any inconsistencies.Then the capacity determination element 118 can adjust one or more ofthe capacity tests and/or system and perform the inconsistent capacitytests again until the first set of capacity test results, second set ofcapacity test results, and third set of capacity test results areconsistent with what is expected. For example, if the second set ofcapacity tests (e.g., spike test) determines a value higher than thecapacity ceiling determined by the first set of capacity tests, then thecapacity determination element 118 can determine that the first set ofcapacity tests, second set of capacity tests, and/or both need to beadjusted because of the inconsistency of results. The first set ofcapacity tests and the second set of capacity tests can be performedagain and adjusted as necessary until the results are consistent withwhat is expected. A fourth set of capacity tests can be performed usingone or more testing parameters (e.g., load volume, ramp up rate)associated with the one or more of first set of capacity tests, thesecond set of capacity tests, and/or the third set of capacity teststhat are calibrated.

In yet another aspect, the capacity determination element 118 can beconfigured to determine a system capacity (e.g., transaction capacityand/or user capacity) based on the first set of capacity test results,the second set of capacity test results, the third set of capacity testresults, a fourth set of capacity test results derived from the fourthset of capacity tests, and/or the like.

In an aspect, a first set of capacity tests (e.g., steady state test)can be implemented by applying a ramp up rate of transactions. Forexample, applying a ramp up rate can comprise continuously addingadditional load to the system. A load can be based on transactions,users, events, connections, and the like. For example, the ramp up ratecan be X transactions per Y seconds, where X and Y can be any positivenumber appropriate for testing purposes. As an example, X can comprise1, 2, 3, 5, 10, 25, 100, and/or the like. Y can comprise 1, 5, 10, 12,20, 15, and/or the like. The ramp up rate can reach a load of Z whichcan be any positive number appropriate for testing purposes. As anexample, Z can be 10, 100, 1000, 10000 transactions and/or the like. Byway of illustration, the ramp up rate can be 1 transaction per 12seconds. Throughout the disclosure, the load will be described as atransaction load, though it should be appreciated that the disclosurecontemplates the use of other loads. The first set of capacity tests cancontinue to apply additional load to the system until the system isnoncompliant with a predefined system requirement (e.g., SLA). Thesystem requirement can comprise predefined performance parameters of thesystem such as latency, processor/memory utilization, capacity,transaction rates, errors, error rates, availability, and/or the like.The transaction load being performed before noncompliance with thesystem requirement can be indicative of a maximum transaction load onthe system also referred to herein as a capacity ceiling. The first setof test results indicates the capacity ceiling because the slow ramp uprate in the steady state test nearly eliminates the ramp up rate as afactor in affecting the capacity ceiling (e.g., maximum transaction loadon the system).

A second set of capacity tests (e.g., a spike test) can be performed byapplying a plurality of predefined transaction loads such as a firsttransaction load, a second transaction load, and so on. The second setof capacity tests can be performed to determine a capacity floor whichis a maximum number of transactions in the least amount of time wherethe performance of the system can comply with the system requirement.The capacity determination element 118 can perform the second set ofcapacity tests on the system. The capacity determination element 118 canapply a first ramp up rate of transactions to the system, which can bethe fastest ramp up rate that the system can perform. The first ramp uprate can be applied to the system until the first transaction load ismet. If the first transaction load is compliant with the systemrequirement, then the second transaction load can be greater than thefirst transaction load. If the first transaction load is noncompliantwith the system requirement, then the second transaction load can beless than the first transaction load. The capacity determination element118 can continue to perform the second set of capacity tests until alesser transaction load is compliant with the system requirement and thenext greater transaction load is noncompliant with the systemrequirement, which indicates that the lesser transaction load is themaximum number of transactions the system can perform in the shortestamount of time (e.g., capacity ceiling).

For example, the capacity determination element 118 can perform thesecond set of capacity tests (spike test) by applying a plurality oftransaction loads (e.g., 1000, 900, 800, 700, 600, or any suitablenumber of transactions) to the system to determine a capacity floor ofthe system. For example, a predefined load of 1000 transactions can beadded to the system quickly (e.g., nearly instantaneously) to determinea maximum transaction load on the system in the shortest time. As anillustration, if adding 1000 transactions results in the system beingnoncompliant with the system requirement, then a smaller number oftransactions can be added to the system (such as 900 transactions) todetermine whether the performance of the system complies with the systemrequirement. If the spike test of 900 transactions complies with thesystem requirement, then 900 transactions can be the capacity floor(e.g., the maximum transaction load on the system in the shortest amountof time). Since transactions cannot be added to the system any fasterthan during the spike test, the spike test results indicate that anyramp up rate that is equal or slower will be in compliance with thesystem requirement for at least 900 transactions. The spike test resultsalso indicate that any transaction load less than 900 transactions,regardless of the ramp up rate, should still comply with the systemrequirement.

A third set of capacity tests (e.g., a transient test) can use one ormore predefined transaction loads with a plurality of ramp up rates. Thethird set of capacity tests can vary the ramp up rate of transactions tothe predefined transaction loads to determine a maximum ramp up rate forthe system. The maximum ramp up rate can be a ramp up rate that is incompliance with a system requirement for the transaction loads betweenthe capacity ceiling and capacity floor. The capacity determinationelement 118 can apply the first ramp up rate to the system until thefirst transaction load is achieved. If the first ramp up rate complieswith the system requirement, then a second ramp up rate can be appliedto the system until first transaction load is achieved. The second rampup rate can be a faster ramp up rate than the first ramp up rate. Thecapacity determination element 118 can determine whether the second rampup rate is in compliance with the system requirement. If the second rampup rate is in compliance with the system requirement, then a third rampup rate that is faster than the second ramp up rate can be applied tothe system until the first transaction load is met and so on until aramp up rate is noncompliant with the system requirement. If the secondramp up rate is noncompliant with the system requirement, then the firstramp up rate is the maximum ramp up rate for the first transaction load.The third set of capacity tests can then perform one or more ramp uprates on a second transaction capacity and so on until the maximum rampup rate for each transaction load is the same for at least twoconsecutive transaction loads. In an aspect, the maximum ramp up rate ofthe first transaction load can be a first ramp up rate applied to thesystem until the second transaction load is achieved. If the first rampup rate fails, a second slower ramp up rate can be applied to the systemuntil the second transaction load is achieved.

As an example, the third set of capacity tests (transient test) can load1000 transactions with a ramp up rate of 1 transaction per 6 seconds, 1transaction per 3 seconds, and the like. The transient test is used todetermine a maximum ramp up rate that should work for all transactionloads between the capacity ceiling and capacity floor. For example, thetransient test can determine that the maximum ramp up rate for 1000transactions is 1 transaction per 1 seconds but the maximum ramp up ratefor 1100 transactions is 1 transaction per 2 seconds and still complieswith the system requirement. The transient test can also show that themaximum ramp up rate for 1200 transactions is 1 transaction per 2seconds. Since the ramp up rate between 1100 transactions and 1200transactions does not change, 1 transaction per 2 seconds is likely tobe in compliance with the system requirement for transaction loads of1100 transactions and greater. The transient test results can be thatthe maximum ramp up rate is 1 transaction per second. The result of thetransient test can be used in conjunction with the capacity ceiling ofthe first set of capacity tests (e.g., steady state test), capacityfloor of the second set of capacity tests (e.g., spike test), and theramp up rate of the first set of capacity tests to determine a firstregion of system capacity.

In an aspect, the first set of capacity tests, the second set ofcapacity tests, the third set of capacity tests, and the fourth set ofcapacity tests can be performed in any order. In an aspect, the methodscan be performed with fewer capacity tests than are described herein. Inan aspect, other capacity tests can be contemplated other than thosedescribed as examples for each of the first set of capacity tests, thesecond set of capacity tests, the third set of capacity tests, and thefourth set of capacity tests (e.g., the steady state tests, the spiketests, transient tests, and soak tests, respectively).

Furthermore, one or more of the first set of capacity tests, the secondset of capacity tests, and the third set of capacity tests can becalibrated based on the first set of capacity test results, the secondset of capacity test results and the third set of capacity test results.If a result of one of the capacity test results contradicts othercapacity test results, then the capacity determination element 118 canmake adjustments (e.g., change ramp up rate) to the system and/orcapacity tests then retest. For example, if the spike test issuccessfully completed for a transaction load greater than the capacityceiling, then either the spike test was performed incorrectly or thecapacity ceiling is incorrect, which means that the steady state testmay need to be adjusted.

As an example, when running a first set of capacity tests (e.g., steadystate tests), a procedure of a steady state test can be followed. Forexample, a number of transactions and a ramp up rate associated with thesteady state test can be set to 10 transactions per minute for 10minutes and wait for 5 minutes then the steady state test can continuethe same ramp up rate for 10 minutes and wait for 5 minutes. The stepsof the procedure can be continued until the performance of the system isnoncompliant with a system requirement. A transaction load beforenoncompliance with the system requirement can be determined as acapacity ceiling and a candidate for system capacity (e.g., maximumtransaction capacity).

As another example, when running a third set of capacity tests (e.g.,transient tests), the procedure of the steady state test can be followedbut with a faster ramp up rate. In certain scenarios where the system isin noncompliance with the system requirement, when the ramp up rate inthe transient test (8 transactions per minute) is slower than the steadystate (10 transactions per minute). Such scenarios can indicate thetransient test was conducted incorrectly and/or the steady state testwas performed incorrectly. Therefore, system configuration and/or testparameters (e.g., ramp up rate, maximum transaction load, waiting time)of the first set of capacity tests and the third set of capacity testscan be calibrated by determination element 118 to make transient testresults coincide with steady state results.

As another example of capacity test calibration, a third set of capacitytests (transient test) and a second set of capacity tests (spike test)can use the same number of transaction loads (e.g., 1000, 900, 800, 700transactions). Where both the spike test and the transient test areusing the same number of transactions, if the spike test can ramp up all1000 transactions in 20 minutes at 50 transactions/minute and thetransient test can ramp up at 40 transactions/minute, the inconsistencycan be an indication that the spike test and/or the transient test areperformed incorrectly and/or an error occurred, because the spike testshould have less bandwidth (e.g., total transactions) than the transienttest due to the burst of transactions. The fastest ramp up rate a spiketest should have is the transient ramp up rate or less since thetransient ramp up rate is the maximum ramp up rate for the given numberof transactions. If the spike test results and the transient testresults are not consistent, configuration of the capacity determinationelement 118 and/or a system configuration (e.g., ramp up rate) can becalibrated.

In an aspect, a fourth set of capacity tests (e.g., a soak test) can beperformed. A soak test can be used to further define and/or verify asystem capacity range determined by the first region defined by thefirst set of capacity test results, the second set of capacity testresults, and the third set of capacity test results. An example of thefirst region and how the first set of capacity test results, the secondset of capacity test results, and the third set of capacity test resultsdefines the first region is illustrated and described in FIG. 4 below.The soak test can use the capacity ceiling determined by the first setof capacity test results and the maximum ramp up rate determined by thethird set of capacity test results as test parameters. Once the capacityceiling is reached by applying the maximum ramp up rate, the transactionload can be set to constant (no more additional load) to determinesystem capacity in terms of transactions per second (TPS). The constanttransaction load can be applied to the system for an extend period oftime (e.g., 12 hours). The fourth set of capacity tests can furtherdefine and/or verify the first region of the system capacity defined bythe first set of capacity tests, the second set of capacity tests, andthe third set of capacity tests. If not verified (the fourth set ofcapacity tests were in noncompliance with the system requirement), thefourth set of capacity tests can be adjusted and further define a secondregion of the first region. The capacity determination element 118 canuse the second region to determine, select, calculate, and/or the likesystem capacity (e.g. calibrated maximum transaction rate) from thesecond region in transactions per second or maximum transaction load.

As an example, the soak test can be performed by using a predefinedratio (e.g., 85%) of the capacity ceiling determined by the first set ofcapacity tests. If the soak test failed with a predefined ratio (e.g.,85%) of the capacity ceiling determined by the first set of capacitytests, the predefined ratio (e.g., 85%) can be calibrated to a smallerratio (e.g., 80%, 75%, etc.). A system capacity (e.g. a maximumtransaction load or ramp up rate) of the system can be determined basedon the first set of capacity test results, the second set of capacitytest results, the third set of capacity test results, and a fourth setof capacity test results of the fourth set of capacity tests. In anaspect, other combinations of capacity test results can be used tocalibrate the first set of capacity tests, second set of capacity tests,third set of capacity tests, and fourth set of capacity tests.

In an aspect, the capacity determination element 118 can be configuredto determine a quantity of active users of a system (e.g., computingdevice 104) and a plurality of user events (e.g., user operations)associated with the quantity of active users. The capacity determinationelement 118 can be configured to determine user event distribution databased on the quantity of active users and the plurality of user eventsassociated with the quantity of active users. User event distributiondata can be categorized such that the user event distribution data isgrouped by a type of user event. For example, a quantity of active userson a specific website can perform a plurality of user events, such asperforming a search, browsing, clicking next page, registering, loggingin, and the like, which each can be categories of user events. Inanother aspect, categorized user event distribution data or the userevent distribution data can be represented in a matrix format. In anaspect, the user event distribution data can comprise a plurality ofuser events, respective counts of the plurality of user events, and apercentage weight of the respective plurality of user events based onthe respective counts. In an aspect, the user event distribution data P(U-DNA) can be determined based on a percentage weight of user eventsfor the quantity of active users. As such, a percentage weight for eachtype of user event can be determined. In an aspect, a user eventdistribution percentage can be determined by categorizing user eventsand calculating percentage weight of each type of user event over totaluser events performed by the quantity of active users. A mathematicalformula to calculate the user event distribution percentage can bewritten as: P (E^(m))=E^((x))/Sum (E⁰˜E^(m)), where P representspercentage, and EX represents user events E¹, E² . . . E^(m), and mrepresents the total number of user events performed by the plurality ofusers.

In an aspect, the capacity determination element 118 can be configuredto determine a user event transaction sequence for each of the quantityof active users. In an aspect, the user event transaction sequence canindicate a sequence of transactions for each of the plurality of userevents performed one at a time. An example of the user event transactionsequence is illustrated below in Table 3.

In an aspect, transaction distribution data can be generated based onthe user event distribution data and the user event transactionsequence. In an aspect, the transaction distribution data can becategorized in the same manner as the user event distribution data. Inan aspect, the transaction distribution data can be formatted as amatrix. The capacity determination element 118 can be configured todetermine a maximum number of active users associated with a systembased on the transaction distribution data of the system. In an aspect,determining a maximum number of users associated with the system basedon the transaction distribution data can comprise multiplying aperformance parameter of the system (e.g., transaction capacitydetermined by the capacity determination element 118) and thetransaction distribution data P (U-DNA). The performance parameter ofthe system can comprise transactions per second (TPS) of the system. Inan aspect, the capacity determination element 118 can access the userevent history 116 for any user event data to determine user capacity ofa system.

In an aspect, the methods and systems can be implemented on a computer201 as illustrated in FIG. 2 and described below. By way of example, theuser device 102 and the computing device 104 of FIG. 1 can be a computeras illustrated in FIG. 2 . Similarly, the methods and systems disclosedcan utilize one or more computers to perform one or more functions inone or more locations. FIG. 2 is a block diagram illustrating anexemplary operating environment for performing the disclosed methods.This exemplary operating environment is only an example of an operatingenvironment and is not intended to suggest any limitation as to thescope of use or functionality of operating environment architecture.Neither should the operating environment be interpreted as having anydependency or requirement relating to any one or combination ofcomponents illustrated in the exemplary operating environment.

The present methods and systems can be operational with numerous othergeneral purpose or special purpose computing system environments orconfigurations. Examples of well-known computing systems, environments,and/or configurations that can be suitable for use with the systems andmethods comprise, but are not limited to, personal computers, servercomputers, laptop devices, and multiprocessor systems. Additionalexamples comprise set top boxes, programmable consumer electronics,network PCs, minicomputers, mainframe computers, distributed computingenvironments that comprise any of the above systems or devices, and thelike.

The processing of the disclosed methods and systems can be performed bysoftware components. The disclosed systems and methods can be describedin the general context of computer-executable instructions, such asprogram modules, being executed by one or more computers or otherdevices. Generally, program modules comprise computer code, routines,programs, objects, components, data structures, and the like thatperform particular tasks or implement particular abstract data types.The disclosed methods can also be practiced in grid-based anddistributed computing environments where tasks are performed by remoteprocessing devices that are linked through a communications network. Ina distributed computing environment, program modules can be located inboth local and remote computer storage media including memory storagedevices.

Further, one skilled in the art will appreciate that the systems andmethods disclosed herein can be implemented via a general-purposecomputing device in the form of a computer 201. The components of thecomputer 201 can comprise, but are not limited to, one or moreprocessors 203, a system memory 212, and a system bus 213 that couplesvarious system components including the processor 203 to the systemmemory 212. In the case of one or more processors 203, the system canutilize parallel computing.

The system bus 213 represents one or more of several possible types ofbus structures, including a memory bus or memory controller, aperipheral bus, an accelerated graphics port, and a processor 203 orlocal bus using any of a variety of bus architectures. By way ofexample, such architectures can comprise an Industry StandardArchitecture (ISA) bus, a Micro Channel Architecture (MCA) bus, anEnhanced ISA (EISA) bus, a Video Electronics Standards Association(VESA) local bus, an Accelerated Graphics Port (AGP) bus, and aPeripheral Component Interconnects (PCI), a PCI-Express bus, a PersonalComputer Memory Card Industry Association (PCMCIA), Universal Serial Bus(USB) and the like. The system bus 213, and all buses specified in thisdescription can also be implemented over a wired or wireless networkconnection and each of the subsystems, including the processor 203, amass storage device 204, an operating system 205, capacity determinationsoftware 206, transaction data 207, a network adapter 208, system memory212, an Input/Output Interface 210, a display adapter 209, a displaydevice 211, and a human machine interface 202, can be contained withinone or more remote computing devices 214 a,b,c at physically separatelocations, connected through buses of this form, in effect implementinga fully distributed system.

The computer 201 typically comprises a variety of computer readablemedia. Exemplary readable media can be any available media that isaccessible by the computer 201 and comprises, for example and not meantto be limiting, both volatile and non-volatile media, removable andnon-removable media. The system memory 212 comprises computer readablemedia in the form of volatile memory, such as random access memory(RAM), and/or non-volatile memory, such as read only memory (ROM). Thesystem memory 212 typically contains data such as transaction data 207and/or program modules such as operating system 205 and capacitydetermination software 206 that are immediately accessible to and/or arepresently operated on by the processor 203.

In another aspect, the computer 201 can also comprise otherremovable/non-removable, volatile/non-volatile computer storage media.By way of example, FIG. 2 illustrates a mass storage device 204 whichcan provide non-volatile storage of computer code, computer readableinstructions, data structures, program modules, and other data for thecomputer 201. For example and not meant to be limiting, a mass storagedevice 204 can be a hard disk, a removable magnetic disk, a removableoptical disk, magnetic cassettes or other magnetic storage devices,flash memory cards, CD-ROM, digital versatile disks (DVD) or otheroptical storage, random access memories (RAM), read only memories (ROM),electrically erasable programmable read-only memory (EEPROM), and thelike.

Optionally, any number of program modules can be stored on the massstorage device 204, including by way of example, an operating system 205and capacity determination software 206. Each of the operating system205 and capacity determination software 206 (or some combinationthereof) can comprise elements of the programming and the capacitydetermination software 206. Transaction data 207 can also be stored onthe mass storage device 204. Transaction data 207 can be stored in anyof one or more databases known in the art. Examples of such databasescomprise, DB2®, Microsoft® Access, Microsoft® SQL Server, Oracle®,mySQL, PostgreSQL, and the like. The databases can be centralized ordistributed across multiple systems.

In another aspect, the user can enter commands and information into thecomputer 201 via an input device (not shown). Examples of such inputdevices comprise, but are not limited to, a keyboard, pointing device(e.g., a “mouse”), a microphone, a joystick, a scanner, tactile inputdevices such as gloves, and other body coverings, and the like These andother input devices can be connected to the processor 203 via a humanmachine interface 202 that is coupled to the system bus 213, but can beconnected by other interface and bus structures, such as a parallelport, game port, an IEEE 1394 Port (also known as a Firewire port), aserial port, or a universal serial bus (USB).

In yet another aspect, a display device 211 can also be connected to thesystem bus 213 via an interface, such as a display adapter 209. It iscontemplated that the computer 201 can have more than one displayadapter 209 and the computer 201 can have more than one display device211. For example, a display device can be a monitor, an LCD (LiquidCrystal Display), or a projector. In addition to the display device 211,other output peripheral devices can comprise components such as speakers(not shown) and a printer (not shown) which can be connected to thecomputer 201 via Input/Output Interface 210. Any step and/or result ofthe methods can be output in any form to an output device. Such outputcan be any form of visual representation, including, but not limited to,textual, graphical, animation, audio, tactile, and the like. The displaydevice 211 and computer 201 can be part of one device, or separatedevices.

The computer 201 can operate in a networked environment using logicalconnections to one or more remote computing devices 214 a,b,c. By way ofexample, a remote computing device can be a personal computer, portablecomputer, smartphone, a server, a router, a network computer, a peerdevice or other network node, and so on. Logical connections between thecomputer 201 and a remote computing device 214 a,b,c can be made via anetwork 215, such as a local area network (LAN) and/or a general widearea network (WAN). Such network connections can be through a networkadapter 208. A network adapter 208 can be implemented in both wired andwireless environments. Such networking environments are conventional andcommonplace in dwellings, offices, enterprise-wide computer networks,intranets, and the Internet.

For purposes of illustration, application programs and other executableprogram components such as the operating system 205 are illustratedherein as discrete blocks, although it is recognized that such programsand components reside at various times in different storage componentsof the computer 201, and are executed by the data processor(s) of thecomputer. An implementation of capacity determination software 206 canbe stored on or transmitted across some form of computer readable media.Any of the disclosed methods can be performed by computer readableinstructions embodied on computer readable media. Computer readablemedia can be any available media that can be accessed by a computer. Byway of example and not meant to be limiting, computer readable media cancomprise “computer storage media” and “communications media.” “Computerstorage media” comprise volatile and non-volatile, removable andnon-removable media implemented in any methods or technology for storageof information such as computer readable instructions, data structures,program modules, or other data. Exemplary computer storage mediacomprises, but is not limited to, RAM, ROM, EEPROM, flash memory orother memory technology, CD-ROM, digital versatile disks (DVD) or otheroptical storage, magnetic cassettes, magnetic tape, magnetic diskstorage or other magnetic storage devices, or any other medium which canbe used to store the desired information and which can be accessed by acomputer.

FIG. 3 is a flowchart illustrating an example method 300 to determine asystem capacity. At step 301, a first set of capacity test results canbe determined by performing a first set of capacity tests on a system.As an example, the system can be a system that provides a service suchas a DVR service, an encoding service, a decoding service, a webservice, and/or other types of services. As another example, the systemcan comprise any transactional application, such as a bankingtransactional application, other online transactional applications, anysystem performance application for a benchmarking test, and the like. Asan example, the first set of capacity tests can comprise a steady statetest. The first set of capacity tests can determine a capacity ceilingsuch as a maximum transaction load that is in compliance with particularnetwork criteria or business driven criteria or, such as a systemrequirement of a service level agreement (SLA), that can be predefined.In an aspect, the system requirement can be based on a specific system,a service and/or an application.

In an aspect, a goal of the first set of capacity tests (e.g., steadystate test) can be to identify the capacity ceiling for the system basedon the system requirement. In an aspect, a steady state test can beimplemented by applying a load (e.g., transaction load) to anapplication, a service, and/or a system with a ramp up rate such thatthe ramp up rate does not affect performance.

In an aspect, the first set of capacity test results can comprise apredefined number of capacity tests, for example, one capacity test, twocapacity tests, or any appropriate number of capacity tests. The firstset of capacity test results can be derived according to one or morepredefined testing parameters (e.g., transaction load, ramp up rate oftransactions) applied to the first set of capacity tests. For example,the first set of capacity tests can be a steady state test. The steadystate test can be performed according to a transaction load with amoderate ramp up rate (e.g., 1 transaction every 6 seconds, 1transaction every 12 seconds, etc.). In an aspect, soaking (e.g.,applying a constant transaction load) can be performed for a predefinedtime period (e.g., soaking at 1000 transactions for 10 minutes) afterramp up to observe resource utilization of a service/system under test(SUT) (e.g., CPU, memory, input, output, latency, and other keyperformance indicators). As an example, a system requirement of an SLAcan require that resource utilizations be maintained within a predefinedamplitude when the resource utilizations are monitored by the SUT. If aresource utilization amplitude is spiky, then this condition can resultin the SUT not being in compliance with the SLA. Thus, the ramp up rateand the soak time can be adjusted until the SUT is in compliance withthe SLA. When the resource utilization amplitude is not spikey duringramp up, then this condition can indicate that the ramp up rate isminimally affecting resource utilization of the SUT. Therefore, thetransaction load can be singled out as the testing parameter thatprimarily affects the resource utilization and can be adjusted todetermine the capacity ceiling.

In an aspect, the first set of capacity tests (e.g., steady state test)can be configured with one or more ramp up rates that are constant(e.g., 1 transaction per 6 seconds, 1 transaction per 12 seconds). In anaspect, a first capacity test of the first set of capacity tests canramp up at a first ramp up rate and soaked at various constant loadvolumes (e.g., transaction loads) repeatedly until noncompliance withthe system requirement to determine the SUT's maximum load volume (e.g.,capacity ceiling). For example, if 2000 transactions can be successfullyperformed before noncompliance with the SLA of a SUT, then 2000transactions can be the SUT's maximum transaction load capacity (e.g.,capacity ceiling). In an aspect, a second capacity test of the first setof capacity tests can ramp up at a second ramp up rate and soaked atvarious constant load volumes (e.g., transaction loads) until the SUT isin noncompliance with the system requirement to verify the maximum loadvolume of the first capacity test and to further define an optimal rampup rate.

At step 302, a second set of capacity test results can be determined byperforming a second set of capacity tests of the system. The second setof capacity tests can comprise a spike test. The second set of capacitytests can determine a maximum number of transactions that the system canprocess in the least amount of time for the performance of the system tostill be compliant with the system requirement. In an aspect, a secondset of capacity tests can be performed by applying a plurality ofpredefined transaction loads (e.g., 1000, 900, 800, 700, 600 users) witha very high ramp up rate (e.g., nearly instantaneously) such as thehighest ramp up rate possible for the system. The highest predefinedtransaction load to comply with the system requirement during the spiketest can indicate a capacity floor. The system capacity can be definedby a region somewhere between the capacity floor, capacity ceiling, andfurther defined by the ramp up rate of the first set of capacity tests.The second set of capacity test results can further verify the boundaryof the capacity ceiling determined at step 301 as explained further instep 304.

In another aspect, the second set of capacity test results can bederived according to one or more predefined testing parameters. Forexample, a spike test can apply a plurality of predefined transactionloads (e.g., 1000, 900, 800, 700, 600 transactions) with a plurality oframp up rates (e.g., 60, 50, 40, 30 transactions/minute). In an aspect,the plurality of predefined transaction loads to maintain current activetransactions can be applied throughout capacity testing. No additionaltransactions can be added. In an aspect, the second set of capacity testresults can comprise a respective predefined number of capacity tests(e.g., 3 tests, 5 tests).

-   -   An example spike test result can comprise:    -   #1: 1000 transactions completed instantaneously with error    -   #2: 900 transactions completed instantaneously without error    -   #3: 800 transactions completed instantaneously without error

In an aspect, one or more ramp up rates from the successful spike testresult can be calculated, if the spike tests have some ramp up time toreach the desired number of transactions. An example ramp up rate cancomprise:

-   -   #2: 56 transactions/minute    -   #3: 57 transactions/minute

One or more ramp up rates can be used for a third set of capacity tests'(e.g., transient test) validation.

At step 303, a third set of capacity test results can be determined byperforming a third set of capacity tests on the system. The third set ofcapacity test results can be in terms of transactions per second (TPS).As an example, the third set of capacity tests can be transient tests.The third set of capacity tests (e.g., transient tests) can comprise aplurality of tests (e.g. 2 tests, 3 tests, 4 tests, 5 tests, etc. . . .). In an aspect, the goal of the transient test can be to identify amaximum ramp up rate for the system requirement. The transient test canbe implemented by applying defined constant transaction loads with aplurality of ramp up rates. In an aspect, the defined constanttransaction loads can be the same transaction loads used for the spiketests of the second set of capacity tests (e.g., 1000, 900, 800, 700,600 transactions). For example, the predefined constant transaction loadcan be 1000 transactions. As an illustration, the plurality of ramp uprates can be:

-   -   #1: 1 transaction/1 second: 1000 transactions completed with        error or not in compliance with the system requirement in 17        minutes (58 transactions/minute)    -   #2: 1 transaction/2 seconds: 1000 transactions completed without        error or in compliance with the system requirement in 34 minutes        (29 transactions/minute)    -   #3: 2 transactions/3 seconds: 1000 transactions completed        without error or in compliance with the system requirement in 25        minutes (40 transactions/minute).

Test #3 can be the maximum ramp up rate because test #3 has the fastestramp up rate that is still in compliance with the system requirement.The ramp up rate of test #3 can then be verified by increasing thetransaction load (e.g. 1100) but using the same ramp up rate (40transactions/minute). If the ramp up rate of 40 transactions/minute for1100 transactions does not cause an error and/or is in compliance withthe system requirement, then the ramp up rate of 40 transactions/minuteis likely be in compliance with the system requirement for any number oftransactions greater than 1100 transactions. Therefore, applying themaximum ramp up rate to the capacity ceiling determined by the first setof capacity tests can further define the system capacity. The systemcapacity can be defined by the capacity floor, capacity ceiling, theramp up rate of the first set of capacity tests, and the maximum ramp uprate of the third set of capacity tests applied to the capacity ceiling.

At step 304, one or more of the first set of capacity tests, the secondset of capacity tests, and the third set of capacity tests can becalibrated based on the first set of capacity test results, the secondset of capacity test results, and the third set of capacity testresults. The one or more of the first set of capacity tests, the secondset of capacity tests, and the third set of capacity tests can becalibrated after verifying respective sets of capacity test results arenot consistent with each other.

In an aspect, the second set of capacity tests can be used to verify thefirst set of capacity test results. For example, if a spike test isperformed successfully at or greater than the capacity ceiling of thefirst set of capacity tests, then this result can indicate that thefirst set of capacity tests were performed incorrectly or the capacityceiling determined by the first set of capacity tests is not a truecapacity ceiling. The system and capacity determination element 118 ofFIG. 1 can adjust the system, the first set of capacity tests, and/orthe second set of capacity tests. One or more of the first set ofcapacity tests and/or second capacity tests can be performed again untileach set of capacity tests are verified to be consistent.

In an aspect, the third set of capacity test results can be used toverify the second set of capacity test results and vice versa. If theramp up rate of the second set of capacity test results used to verifythe capacity floor is ever greater than the third set of capacity testresults, then this condition can indicate that the second set ofcapacity test results and/or the third set of capacity test results arenot correct, because the third set of capacity tests determines themaximum ramp up rate. If a ramp up rate is greater in another capacitytest, then this condition indicates there was an error andconfiguration/calibration by the capacity determination element 118 ofthe system and/or test parameters (e.g., ramp up rate) of the third setof capacity tests and second set of capacity tests can be performed.

At step 305, a fourth set of capacity tests can be performed using oneor more test parameters associated with one or more of the first set ofcapacity tests, the second set of capacity tests, and the third set ofcapacity tests. A fourth set of capacity test results can be derivedfrom the fourth set of capacity tests.

In an aspect, a soak test can be performed by applying constanttransaction load determined by the first set of capacity test results(e.g., capacity ceiling) for a predefined time period (e.g., 10 minutes,1 hour, 3 hours, 12 hours, and/or the like) without increasing thetransaction load. In an aspect, before the fourth set of capacity testsare performed, the transaction load and ramp up rates to determine thesystem capacity can be within a large range of possibilities. The fourthset of capacity tests can further define a system capacity rangedetermined by the first set of capacity tests, the second set ofcapacity tests, and the third set of capacity tests, thereby narrowingdown the large range. The fourth set of capacity tests can further beused to verify the large range determined by the first set of capacitytests, the second set of capacity tests, and the third set of capacitytests.

In an aspect, a goal of the soak test can be to obtain an optimal systemcapacity for a system requirement. The system capacity can be based onthe system requirement and the region of possible soak capacitiesdefined by the soak test. The fourth set of capacity test results can bederived based on one or more test parameters (e.g., transaction load,ramp up rate) associated with one or more of the first set of capacitytests, the second set of capacity tests, and the third set of capacitytests that are calibrated. The soak test can be implemented by applyinga first soak test at a transaction load that is greater than that of thecapacity ceiling. The capacity ceiling can be a result of the first setof capacity tests. The soak test can further be implemented by applyinga second soak test that is less than that of the capacity ceiling. Forexample the first soak test and second soak test can be performed at115% and 85%, respectively, of the capacity ceiling. The fourth set ofcapacity tests (e.g., soak test) can be performed by applying atransaction load of 85% of the capacity ceiling (e.g., 85% of 2000=1700transactions).

The maximum ramp up rate determined by the third set of capacity testscan be used to reach the transaction load of a soak stage. Once a SUTreaches the transaction load (e.g., 1700 transactions), the SUT can bemaintained (e.g., soaked) at the transaction load (e.g., 1700transactions) for an extended amount of time such as 1 hour, 2 hours, 12hours, or however long is necessary to generate accurate results.Expected results of the fourth set of capacity tests, during ramp up,can be performance of the SUT to be compliant with the systemrequirement. Expected results of the fourth set of capacity tests duringthe soak stage with a transaction load at 85% of the capacity ceilingcan be the performance of the SUT to be compliant with the systemrequirement. If any of the expected results are not satisfied, one ormore test parameters such as ramp up rate and/or percentage of the totalcapacity (e.g., 80% instead of 85%) can be adjusted in the fourth set ofcapacity tests until the expected results are satisfied.

In another aspect, the soak test can be performed at the capacityceiling (e.g. 100%) to verify the SUT can be in compliance with thesystem requirement at the capacity ceiling. In another aspect, the soaktest can be performed at 115% of the capacity ceiling to verify that thecapacity ceiling is a true capacity ceiling and that the first set ofcapacity test results are correct. If at 115% the SUT is in compliancewith the system requirement during the fourth set of capacity tests,then this result can indicate the first set of capacity tests were notperformed correctly or vice versa, then calibration by the capacitydetermination element 118 of the system and/or test parameter (e.g.,ramp up rate) can be performed.

At step 306, a system capacity can be determined based on the first setof capacity test results, the second set of capacity test results, thethird set of capacity test results, and the fourth set of capacity testresults. As an example, determining the system capacity can comprisedetermining one or more of a maximum capacity, a maximum transactionload, a maximum transaction ramp up rate, a maximum system resourceutilization (e.g., CPU, system memory), a maximum number of activeusers, and an optimal region for the system requirement. See FIG. 4 foran example of system capacity determination.

FIG. 4 is a graph 400 illustrating capacity test results of a system anddetermination of a system capacity such as transaction capacity, or rampup rate (e.g., transactions per second) based on the capacity testresults of the system. The graph 400 illustrates the number oftransactions being performed (y-axis) with respect to time (x-axis). Inan aspect, a first set of capacity tests can be performed. The first setof capacity tests can be a steady state test. Steady state test results,represented by steady state plot 405, can create a general region forsystem capacity. The steady state test can gradually ramp up untilsystem performance does not comply with a system requirement.

As an example, the results of steady state plot 405 show the steadystate test ramping up at a steady ramp up rate but once the steady stateplot 405 reaches 2200 transactions, performance of the system is not incompliance with a network parameter, a business parameter such as asystem requirement of an SLA. The capacity determination element 118 candetermine that 2200 is a maximum capacity of transactions (e.g.,capacity ceiling 407) for the system based on when the noncomplianceoccurred and what transaction load was on the system before thenoncompliance of the system requirement. The ramp up rate used for thesteady state test to reach the capacity ceiling 407 is not assignificant with the steady state test because a slow ramp up rate upnearly eliminates the ramp up rate as a factor that can affect thesystem to be in noncompliance of the system requirement. As a result ofthe steady state plot 405, the capacity determination element 118 candetermine that an region (e.g., area) defined below 2200 transactionsand to the right of the steady state plot 405 is a functional systemcapacity (e.g., all ramp up rates and transaction loads within theboundary can be performed without being noncompliant with an systemrequirement). The region to the left of the steady state plot 405 and2200 transactions or less can be a system capacity candidate. Anytransaction load regardless of ramp up rate above 2200 transactionsshould be noncompliant with the system requirement.

Possibilities of the system capacity can be further defined by a secondset of capacity tests. The second set of capacity tests can be a set ofspike tests. Spike tests can put immediate transaction loads on thesystem to determine the maximum number of transactions to be completedin compliance with an system requirement in the shortest time. In theexample of graph 400, the capacity determination element 118 couldinitiate several different transaction loads of a spike test todetermine whether any of the transaction loads meet the systemrequirement. For example, the performance capacity engine can initiatetransaction loads of 1700, 1600, 1500, 1400, and 1300 transactions. Thegraph 400 indicates that, for the system, only spike tests of 1300transactions and 1400 transactions (denoted by spike plot 410 a andspike plot 410 b respectively) can meet the system requirement. Spiketests of 1700, 1600, and 1500 transactions (not illustrated in plot 400)did not comply with the system requirement. The capacity determinationelement 118 can determine that any transaction load below 1400transactions should comply with the system requirement no matter whatthe ramp up rate is since the spike test is ramped up nearlyinstantaneously and the transaction load of 1400 is in compliance withthe system requirement. As a result, the capacity determination element118 can determine a minimum transaction load of the system capacityrepresented by a capacity floor 412 that is the transaction load of1400. Therefore, the system capacity candidate is now defined, bounded,and/or the like by the region (e.g., area) formed by the capacityceiling 407, and a capacity floor 412 and steady state ramp up rate ofsteady state plot 405.

Next, the system capacity can be further defined, bounded, and/or thelike by a third set of capacity tests. In an aspect, the third set ofcapacity tests can be transient tests. The transient tests can be usedto find a more optimal ramp up rate that is between the nearlyinstantaneous ramp up rate for the spike tests and the slow ramp up rateof the steady state test. In an aspect, the third set of capacity testscan determine a maximum ramp up rate that can obtain all transactionloads between the capacity ceiling 407 and the capacity floor 412.Defined transaction loads can be used for the transient tests and theramp up rate can be adjusted, varied, and/or the like to determine thefastest ramp up rate the system can handle to achieve the definedtransaction loads and still comply with the system requirement.

In the example of graph 400, the capacity determination element 118 canuse the same defined transaction loads as the spike tests (e.g., 1700,1600, 1500, 1400, and 1300 transactions). Since transaction loads 1400and 1300 are known to comply with the system requirement for the spiketests, the transaction loads 1400 and 1300 do not need to be tested withthe transient test since the ramp up rate of the transient test would bethe same as the ramp up rate of the spike test (nearly instantaneous).In other aspects, the transaction loads 1400 and 1300 can be tested withthe transient tests to verify and calibrate the spike tests.

The capacity determination element 118 can perform transient tests onthe transaction loads of 1500, 1600, and 1700 (transient graphs 415 a,415 b, and 415 c, respectively). A plurality of ramp up rates (e.g., 1transaction per 1 second, 2 transactions per 3 seconds, 1 transactionper 2 seconds) can be used to determine the maximum ramp up rate foreach transaction load. As illustrated in graph 400 the maximum ramp uprate of transient plot 415 a is greater than the ramp up rate oftransient plot 415 b and transient plot 415 c as indicated by thegreater slope to transient plot 415 a. Also indicated in graph 400, theramp up rates of transient plot 415 b and transient plot 415 c are thesame, which can indicate that the ramp up rate of 415 b is a maximumramp up rate. The maximum ramp up rate can be defined as any number oftransaction loads higher (or less) than 1600 transactions that can usethe ramp up rate of transient plot 415 b and still comply with thesystem requirement. Therefore, the capacity determination element 118can determine the maximum ramp up rate is that of transient plot 415 b.The capacity determination element 118 can also determine that thesystem capacity is now defined, bounded, and/or the like by the regionformed by capacity ceiling 407, the capacity floor 412, the steady stateramp up rate of steady state plot 405, and a transient ramp up rate plot416, which is the maximum ramp up rate to the transaction load of thecapacity ceiling 407. In an aspect, depending on the system requirementand whether the ramp up rate is considered higher priority than amaximum transaction load, the transient ramp up rate plot 416 could beat a greater ramp up rate such as transient plot 415 a. However, thecapacity ceiling 407 of the maximum transaction load could decrease. Afirst region 418 generally represents the area defined, bounded, and/orthe like by the capacity ceiling 407, the capacity floor 412 the steadystate ramp up rate of steady state plot 405, and a transient ramp uprate plot 416.

In an aspect, a fourth set of capacity tests can be performed by thecapacity determination element 118. The fourth set of capacity tests canbe used to further define the system capacity or verify the systemcapacity of the first region 418. The fourth set of capacity tests canbe soak tests. Soak tests can use the maximum ramp up rate of thetransient test results to ramp up to a constant transaction load andthen remain at that constant transaction load for an extended period oftime to determine whether the system complies with the systemrequirement. For example, the constant transaction load can be placed onthe system for 12 hours or more. For example, in graph 400 a soak testcan be performed represented by soak plot 420 a. This soak test can beused to confirm that the capacity ceiling is capacity ceiling 407. Thesoak test represented by soak plot 420 a is ramped up at the maximumramp up rate determined by the transient test results and to atransaction load greater than that of capacity ceiling 407. In graph400, soak plot 420 a does not comply with the system requirement whichindicates that capacity ceiling 407 is likely or close to the maximumtransaction load. In another example, a second soak test can beperformed at the capacity ceiling 407. As illustrated in graph 400 atsoak plot 420 b, the second soak test at the capacity ceiling 407 alsofails. This could indicate that the ramp up rate or the capacity ceiling407 is not optimal. The capacity determination element 118 can adjustthe transaction load for a third soak test as in soak plot 420 c and/orthe capacity determination element 118 could adjust the ramp up rate fora fourth soak test as in soak plot 425 b. In graph 400, both the soakplot 420 c and soak plot 425 b represent soak tests that comply with thesystem requirement. From the soak tests, a second region 430 can befurther defined from the first region 418 as a region of the systemcapacity candidate.

The capacity determination element 118 can determine the system capacityat a system capacity point 435 as a function of time within the secondregion 430 to determine the maximum transactions per second. Thecapacity determination element 118 can determine the system capacitypoint 435 based on the system requirement which can rank parameters ofthe system as more critical than others. For example, the capacitydetermination element 118 can choose the system capacity point 435 to beone of a center of the second region 430, a maximum transaction load ofthe second region 430, a minimum transaction load of the second region430, a least amount of time of the second region 430, a greatest amountof time of the second region 430, a point on a plot line (e.g. soak plot420 c), and the like. In an example, the capacity determination element118 can select, determine, calculate, and/or the like the systemcapacity point 435 in graph 400 to be a transaction load of 2000transactions at a ramp up rate of 2000 transactions per 10 minutes whichequals approximately 3 transactions per second. Using a plurality ofdifferent system capacity tests as described can reduce the number oftests to determine system capacity and increase accuracy of the systemcapacity test results.

FIG. 5 is a flowchart illustrating another example method 500. In anaspect, the example method 500 can be used to determine system capacityin terms of maximum number of active users based on system capacity(e.g., transaction capacity and ramp up rate of transactions) determinedby the example method 300. The maximum number of active users can bedetermined based on a plurality of user events driven by a quantity ofactive users and associating each of the plurality of user events withone or more transactions.

At step 501, a quantity of active users of a system and a plurality ofuser events associated with the quantity of active users can bedetermined. In an aspect, the plurality of user events associated withthe quantity of active users can be categorized. For example, a quantityof active users on a specific website can perform a plurality of userevents, such as performing a search, browsing, clicking next page,registering, logging in, and the like. As another example, a quantity ofactive users on a DVR service can perform a plurality of user events,such as deleting recordings, one-time recording, erasing recordings,series recordings, cancelling recordings, and undeleting recordings, andthe like.

In an aspect, determining a quantity of active users of a system and aplurality of user events associated with the quantity of active userscan comprise accessing a user event history data of the system (e.g.,user event history 116). In an aspect, user event history data (e.g.,user event data for past 7-14 days) of the quantity of active users canbe collected by a log analyzer (e.g., Splunk).

At step 502, user event distribution data can be determined based on thequantity of active users and the plurality of user events associatedwith the quantity of active users. User event distribution data can becategorized such that the user event distribution data is grouped by atype of user event. For example, a quantity of active users on aspecific website can perform a plurality of user events, such asperforming a search, browsing, clicking next page, registering, loggingin, and the like, which each can be categories of user events. In anaspect, the user event distribution data can be determined based ondetermining a percentage weight of user event categories for thequantity of active users. Accordingly, the percentage weight for eachuser event category can be calculated. In an aspect, the user eventdistribution data can be formatted as a matrix. An example of the userevent distribution data for a DVR service is shown in Table 1. A userevent distribution percentage weight can be generated by categorizinguser events and calculating percentage weight of each category of userevent over total user events performed by the quantity of active users.A mathematical formula to calculate the percentage weight of the userevent categories can be written as: P (E^(m))=E^((x))/Sum (E⁰˜E^(m)),wherein P represents percentage, and EX represents user events E¹, E² .. . E^(m), and m represents total number of user events.

TABLE 1 count User Event (User Event) Percentage 1 Delete Recordings966818 53% 2 One Time Recording 403568 22% 3 Erase Recordings 255838 14%4 Series Recordings 130752  7% 5 Cancel Recordings 56365  3% 6 UndeleteRecordings 13640  1%

A user event can be a user operation. A user event can correspond to oneor more predefined transactions. For example, a user event cancorrespond to a key transaction (KT). A key transaction can be used toidentify a transaction that is the user event when the user eventperforms more than one transaction. In addition, a user event cancorrespond to one or more non-key transactions. As an illustration, a“one-time recording” user event can correspond to a key transaction of“record one shot.” The “one-time recording” user event can also generate2.5 “Get Recordings” transactions, and one “Get Recordings List” for atotal of 4.5 transactions for one user event. Table 2 illustrates anexample of a plurality of user events and respective predefined numberof transactions associated with the plurality of user events.

TABLE 2 User Event Transactions count 1 Delete Recording DeleteRecordings (KT) 1 2 One-Time Recording Record one shot (KT) 1 GetRecordings 2.5 Get Recordings List 1 3 Erase Recording Erase Recordings(KT) 1 Get Recordings 1 4 Series Recording Add Series (KT) 1 RetrieveSeries 3 Get Recordings 1 Get Recordings List 1 5 Cancel RecordingCancel schedules (KT) 1 Get Recordings 1 Get Recordings List 1 6Undelete Recording Undelete Recordings (KT) 1

At step 503, a user event transaction sequence for each of the quantityof active users can be determined based on the quantity of active usersand the plurality of user events associated with the quantity of activeusers. The user event transaction sequence can be referred as U-DNA. Inan aspect, the user event transaction sequence for each of the quantityof active users can be determined by associating each of the pluralityof the active user events with one or more predefined number oftransactions. A mathematical formula to calculate a user eventtransaction sequence for each of the quantity of active users can bewritten as U-DNA=E⁰T₁, E⁰T₂, E⁰T₃ . . . E⁰T_(n); E¹T₁, E¹T₂, E¹T₃ . . .E¹ T_(n); E^(m)T₁, E^(m)T₂, E^(m)T₃ . . . E^(m)T_(n), wherein Erepresents a user event, T represents a transaction, n represents atotal number of types of transactions per event and m represents a totalnumber of events. The user event transaction sequence can indicate asequence of transactions for each of the plurality of user eventsperformed one at a time.

In an aspect, determining an event transaction sequence for each of thequantity of active users can be based on a predefined number of users.Table 3 illustrates an example user event transaction sequence U-DNAbased on 100 active users. A total transaction count with 100 activeusers caused by a plurality of user events are shown according to userevent categories.

TABLE 3 # of Total events for Transaction transaction 100 active countper count User Event users Transactions user event (232) 1 Delete 53Delete Recordings 1 53 Recordings 2 One Time 22 Record one shot 1 22Recording Get Recordings 2.5 55 Get Recordings List 1 22 3 Erase 14Erase Recordings 1 14 Recordings Get Recordings 1 14 4 Series 7 AddSeries 1 7 Recordings Retrieve Series 3 21 Get Recordings 1 7 GetRecordings List 1 7 5 Cancel 3 Cancel schedules 1 3 Recordings GetRecordings 1 3 Get Recordings List 1 3 6 Undelete 1 Undelete 1 1Recordings Recordings

At step 504, transaction distribution data can be determined, generated,and/or the like based on the user event distribution data obtained atstep 502 and the user event transaction sequence for each of thequantity of active users obtained at step 503. In an aspect, thetransaction distribution data can be determined by aggregatingtransactions into categories and dividing each aggregated transactioncategory by total transactions. In an aspect, the transactiondistribution data can be formatted as matrix. Table 4 illustrates anexample transaction distribution data based on 100 active users. Theuser event distribution data can be generated by aggregating thetransactions into categories (e.g., 9 categories). For example, all ofthe “Get Recording” transactions can be aggregated into a categoryregardless of what user event generated the “Get Recording” transaction.Then each aggregated transaction category can be divided by the totalnumber of transactions (e.g., 232 transactions). The transactiondistribution data can be written as percentage weighting P (U-DNA).

TABLE 4 Percentage Transactions Count Weighting 1 Get Recordings 7934.05% 2 Get Recordings List 32 13.79% 3 Retrieve Series 21  9.05% 4Record One Shot 22  9.48% 5 Delete Recordings 53 22.84% 6 EraseRecordings 14  6.03% 7 Add Series 7  3.02% 8 Cancel Schedules 3  1.29% 9Undelete Recordings 1  0.43%

At step 505, a maximum number of active users associated with the systemcan be determined based on the transaction distribution data. In anaspect, determining an estimated maximum number of users associated withthe system based on the transaction distribution data can comprisemultiplying a performance parameter of the system and the transactiondistribution data P (U-DNA). The performance parameter of the system cancomprise a system capacity of ramp up rate of transactions per second(TPS). The performance parameter (e.g., system capacity) of the systemcan be determined by the example method 300 of FIG. 3 .

Table 5 illustrates a maximum number of users (“estimated active users”)associated with a DVR service in each key transaction (KT) category. Themaximum number of users for each key transaction (KT) category can becalculated using U=Sum (KTs) of (TPS*P (U-DNA)), where U is the maximumnumber of users associated with the system. Key transactions (KT) can belabeled such that the maximum number of users in key transactions can bedetermined. The maximum number of users in each of the key transaction(KT) categories can be added, as shown in Table 5. TPS can be atransaction capacity of a DVR service. TPS can be determined by themethod 300 described in FIG. 3 . As an example, TPS=300 is used as anexample in Table 5. The system capacity of maximum number of activeusers can be 129.

TABLE 5 Transaction Estimated Percentage Per Second Active TransactionsCount Weighting (TPS = 300) users 1 Get Recordings 79 34.05% 102 0 2 GetRecordings List 32 13.79% 41 0 3 Retrieve Series 21  9.05% 27 0 4 RecordOne Shot (KT) 22  9.48% 28 28 5 Delete Recordings (KT) 53 22.84% 69 69 6Erase Recordings (KT) 14  6.03% 18 18 7 Add Series (KT) 7  3.02% 9 9 8Cancel Schedules (KT) 3  1.29% 4 4 9 Undelete Recordings (KT) 1  0.43% 11 Total 129

FIG. 6 is a flowchart of an example method 600. At step 601, a quantityof active users of a system and a plurality of user events associatedwith the quantity of active users can be determined. In an aspect, theplurality of user events associated with the quantity of active userscan be categorized. For example, a quantity of active users on aspecific website can perform a plurality of user events, such asperforming a search, browsing, clicking next page, registering, loggingin, and the like. As another example, a quantity of active users on aDVR service can perform a plurality of user events, such as deletingrecordings, one-time recording, erasing recordings, series recordings,cancelling recordings, undeleting recordings, and the like.

In an aspect, determining a quantity of active users of a system and aplurality of user events associated with the quantity of active userscan comprise accessing a user event history of the system (e.g., userevent history 116). As an example, user event history data (e.g., userevent data for the past 7-14 days) of the quantity of active users canbe received by a log analyzer (e.g., Splunk).

At step 602, user event distribution data (CED) can be determined basedon the quantity of active users of the system and the plurality ofactive user events associated with the quantity of active users. In anaspect, the CED can be categorized. In an aspect, the CED can beformatted as a matrix. In an aspect, the CED can indicate that eachactive user event can correspond to multiple transactions. The CED canbe a summary tabulated from active user events, transaction names, andtransaction counts associated with the active user events.

In an aspect, the user event distribution data can be determined basedon determining a percentage weight of user events for the quantity ofactive users. The percentage weight of the user event distribution canbe generated by categorizing user events and calculating percentageweight of each type of user event over total user events performed bythe quantity of active users. A mathematical formula to calculate thepercentage weight of the user events can be written as: P(E^(m))=E^((x))/Sum (E⁰˜E^(m)), where P represents percentage, and EXrepresents user events E¹, E² . . . E^(m), and m represents total numberof user events.

A user event can be a user operation. A user event can correspond to oneor more predefined transactions. For example, a user event cancorrespond to a key transaction (KT). A key transaction can be used toidentify a transaction that represents the user event when a user eventcomprises more than one transaction. In addition, a user event cancorrespond to one or more non-key transactions. For example, a “one-timerecording” user event can correspond to a key transaction of “record oneshot.” The “one-time recording” user event can also generate 2.5 “GetRecordings” transactions, and one “Get Recordings List” for a total of4.5 transactions for one user event.

At step 603, at least one performance parameter of the system capacitycan be determined according to the user event distribution data. As anexample, the at least one performance parameter can be in terms oftransaction per seconds (TPS). The at least one performance parameter ofthe system can be determined based on one or more of capacity tests,such as steady state test, spike test, transient test, soak test, and/orthe like. The at least one performance parameter can be determined bythe example method 300 of FIG. 3 . The at least one performanceparameter can comprise a system capacity.

At step 604, an event transaction sequence for each of the quantity ofactive users can be determined. The event transaction sequence can bebased on the quantity of active users of the system and the plurality ofuser events associated with the quantity of active users. The eventtransaction sequence can be referred as U-DNA. In an aspect, the usertransaction sequence for each of the quantity of active users can bedetermined by associating each of the plurality of the active userevents with one or more predefined number of transactions. Amathematical formula to calculate an event transaction sequence for eachof the quantity of active users can be written as U-DNA=E⁰T₁, E⁰T₂, E⁰T₃. . . E⁰T_(n); E¹ T₁, E¹ T₂, E¹T₃ . . . E¹T_(n); E^(m)T₁, E^(m)T₂,E^(m)T₃ . . . ET_(n), wherein E represents a user event, T represents atransaction, n represents a total number of transactions per event, andm represents a total number of user events. The event transactionsequence can indicate a sequence of transactions for each of theplurality of user events performed one at a time. In an aspect,determining an event transaction sequence for each of the quantity ofactive users can be based on a predefined number of users.

In an aspect, transaction distribution data can be generated based onthe user event distribution data and the event transaction sequence foreach of the quantity of active users. In an aspect, the transactiondistribution data can be determined by aggregating transactions intocategories and dividing each aggregated transaction category by totaltransactions. The user event distribution data can be generated byaggregating the matching transactions from different user events intocategories and dividing each aggregated transaction category by totalnumber of transactions. The transaction distribution data can be writtenas percentage weighting P (U-DNA). In an aspect, the transactiondistribution data can be formatted as a matrix.

At step 605, a maximum number of users associated with the system can bedetermined based on the user event distribution data, the eventtransaction sequence, and the at least one system capacity parameter. Inan aspect, determining a maximum number of users associated with thesystem based on the transaction distribution data can comprisemultiplying a performance parameter of the system determined at step 603and the transaction distribution data P (U-DNA) determined at step 604.In an aspect, a maximum number of users for each key transactioncategory can be calculated using U=Sum (KTs) of (TPS*P (U-DNA)), where Uis the maximum number of users associated with the system. Keytransactions (KT) can be labeled such that a maximum number of users inkey transactions can be determined. The maximum number of users in eachof the key transaction categories can be added.

The methods and systems can employ artificial intelligence (AI)techniques such as machine learning and iterative learning. Examples ofsuch techniques include, but are not limited to, expert systems, casebased reasoning, Bayesian networks, behavior based AI, neural networks,fuzzy systems, evolutionary computation (e.g. genetic algorithms), swarmintelligence (e.g. ant algorithms), and hybrid intelligent systems (e.g.Expert inference rules generated through a neural network or productionrules from statistical learning).

While the methods and systems have been described in connection withpreferred embodiments and specific examples, it is not intended that thescope be limited to the particular embodiments set forth, as theembodiments herein are intended in all respects to be illustrativerather than restrictive.

Unless otherwise expressly stated, it is in no way intended that anymethod set forth herein be construed as requiring that its steps beperformed in a specific order. Accordingly, where a method claim doesnot actually recite an order to be followed by its steps or it is nototherwise specifically stated in the claims or descriptions that thesteps are to be limited to a specific order, it is in no way intendedthat an order be inferred, in any respect. This holds for any possiblenon-express basis for interpretation, including: matters of logic withrespect to arrangement of steps or operational flow; plain meaningderived from grammatical organization or punctuation; the number or typeof embodiments described in the specification.

It will be apparent to those skilled in the art that variousmodifications and variations can be made without departing from thescope or spirit. Other embodiments will be apparent to those skilled inthe art from consideration of the specification and practice disclosedherein. It is intended that the specification and examples be consideredas exemplary only, with a true scope and spirit being indicated by thefollowing claims.

The invention claimed is:
 1. A method comprising: receiving, at acomputing device, an indication of a plurality of user events associatedwith a plurality of users of a network; determining, based onassociating each user event of the plurality of user events with anumber of transactions, a user event transaction sequence for each userof the plurality of users; and determining, based on the user eventtransaction sequence, a user capacity for the network.
 2. The method ofclaim 1, wherein each user event of the plurality of user eventscomprises at least one of: a user command associated with a user of thenetwork or an action associated with the user of the network.
 3. Themethod of claim 1, wherein each user event of the plurality of userevents comprises at least one transaction, and wherein the user eventtransaction sequence comprises a sequence of the at least onetransaction.
 4. The method of claim 1, wherein user event distributiondata comprises a count of the plurality of user events for each user ofthe plurality of users and a percentage weight of the count of theplurality of user events for each user of the plurality of users.
 5. Themethod of claim 4, wherein determining the user capacity for the networkcomprises: determining, based on the percentage weight of the count ofthe plurality of user events, a maximum quantity of transactions thenetwork can perform at a given time.
 6. The method of claim 1, whereinthe user capacity for the network comprises a maximum quantity of usersthe network can support while maintaining at least one of: a networkrequirement or a network threshold.
 7. A method comprising: receiving,at a computing device, an indication of a plurality of user eventsassociated with a plurality of users of a network; determining, based onthe plurality of user events and the plurality of users, user eventdistribution data; determining, based on the user event distributiondata and a user event transaction sequence for each user of theplurality of users, a count of the plurality of user events for eachuser of the plurality of users; and determining, based on the count ofthe plurality of user events for each user of the plurality of users, auser capacity for the network.
 8. The method of claim 7, wherein eachuser event of the plurality of user events comprises at least one of: auser command associated with a user of the network or an actionassociated with the user of the network.
 9. The method of claim 7,wherein each user event of the plurality of user events comprises atleast one transaction.
 10. The method of claim 7, wherein the user eventtransaction sequence is based on associating each user event of theplurality of user events with a number of transactions.
 11. The methodof claim 7, wherein the user capacity for the network comprises amaximum quantity of users the network can support while maintaining atleast one of: a network requirement or a network threshold.
 12. Themethod of claim 11, wherein the network requirement comprises at leastone of: a latency, a memory utilization, a transaction rate, or an errorrate, and wherein the network threshold comprises at least one of: aquantity of transactions, the transaction rate, or a quantity of users.13. The method of claim 7, wherein determining the count of theplurality of user events for each user of the plurality of userscomprises: determining, based on the user event distribution data andeach user event transaction sequence, a percentage weight of the countof the plurality of user events for each user of the plurality of users.14. A method comprising: receiving, at a computing device, an indicationof a plurality of user events associated with a plurality of users of anetwork; determining, based on the plurality of user events and at leastone transaction associated with each user event of the plurality of userevents, transaction distribution data; and determining, based on theplurality of user events and the transaction distribution data, a usercapacity for the network.
 15. The method of claim 14, wherein a userevent transaction sequence is based on associating each of the pluralityof user events with a number of transactions.
 16. The method of claim14, wherein determining the user capacity for the network comprises:determining, based on the plurality of user events, user eventdistribution data; and determining, for each user of the plurality ofusers, a user event transaction sequence.
 17. The method of claim 16,wherein the user event distribution data comprises a count of theplurality of user events for each user of the plurality of users and apercentage weight of the count of the plurality of user events for eachuser of the plurality of users.
 18. The method of claim 17, whereindetermining the user capacity for the network comprises: determining,based on the percentage weight of the count of the plurality of userevents for each user of the plurality of users, a maximum quantity oftransactions the network can perform at a given time.
 19. The method ofclaim 14, wherein at least one user event of the plurality of userevents or a user event transaction sequence for each user of theplurality of users is associated with an application in communicationwith the network.
 20. The method of claim 16, wherein determining theuser event transaction sequence comprises: multiplying a number of userevents of the plurality of user events and the at least one transactionassociated with each user event of the plurality of user events.